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Abstract 

In visual cognition, illusions help elucidate certain intriguing latent perceptual functions of 
the human vision system, and their proper mathematical modeling and computational simulation 
are therefore deeply beneficial to both biological and computer vision. Inspired by existent prior 
works, the current paper proposes a first-order energy-based model for analyzing and simulating 
illusory contours. The lower complexity of the proposed model facilitates rigorous mathematical 
analysis on the detailed geometric structures of illusory contours. After being asymptotically 
approximated by classical active contours, the proposed model is then robustly computed using 
the celebrated level-set method of Osher and Sethian (J. Comput. Phys., 79:12-49, 1988) with 
a natural supervising scheme. Potential cognitive implications of the mathematical results are 
addressed, and generic computational examples are demonstrated and discussed. 

Keywords: illusion, illusory contours, energy model, local stability, contour decomposition, 
real and imaginary, kinks, spans, turns, level-set method, supervision. 

1 Introduction 



In System Theory [22], input-output analysis has been a major tool for partial or complete identifi- 
cation of black-box systems. In cognitive vision science, the study of various visual illusions follows 
exactly the same spirit. Cognitive scientists have designed numerous intriguing inputs of image 
signals, so that the distorted or transformed outputs (as reported by an average human observer) 
can help reveal some crucial latent properties of the human vision system (see, e.g., the remarkable 
works of Adelson 1, Knill and Kersten |14|ll6j. and Kanizsa |llj). Illusory contours are such a well 
known class of visual illusions, and the current paper develops a mathematical model to character- 
ize, analyze, and simulate generic illusory contours. Our work has been closely inspired by many 
existent modeling works, especially by Sarti, Malladi, and Sethian and Zhu and Chan PUIETT] . 

Fig. Q shows two examples of illusory contours known as Kanizsa triangle and square HH 
I3U| . The illusory or imaginary white triangle and square pop out almost instantly to a normal 
observer. The human vision system is capable of interpolating non-existent edges and making 
meaningful visual organization of both the real and imaginary edge segments. From the viewpoints 
of modeling and computation, such illusory perception has also been called segmentation with 
missing boundaries [21] > an d is closely related to some earlier works (e.g., [1H\ ll^j). 

Qualitatively, the perception of illusory contours can be deciphered by the Gestalt frame- 
work jl()l lllj . In a typical example of illusory contours, the illusory target objects share the 
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(a) (b) 

Figure 1: Two classical examples of illusory contours: Kanizsa's triangle and square. 

same gray level as their background and become invisible to ordinary detectors. The illusory tar- 
gets however do leave their footprints, mainly through the cues of occlusion, e.g., the visible corners 
in Fig. ^ An average human vision system can organize these cues and develop a so-called 2.1-D 
sketch of the scene fF^, i.e., to separate and complete objects in ordered (according to the relative 
depth to a viewer) multiple layers. The multi-layer sketch of Kanizsa's triangle is displayed in 
Fig. |21 for example. We also refer the reader to the recent work of Shen (22] on an abstract study 
of the occlusion phenomenon and 2.1-D models. 
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(b) 
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Figure 2: Multi-layer decomposition of Kanizsa's triangle: (a) image of Kanizsa's triangle, (b) far 
background, (c) three disks in the middle range, and (d) the illusory triangle closest to a viewer. 

Quantitatively, however, it is a nontrivial task to properly model and compute illusory contours. 
There are two notable recent works that have primarily influenced the present work in developing 
plausible mathematical models and robust computational schemes. 

In [21], Sarti, Malladi, and Sethian first proposed a variational-PDE model for computing 
illusory contours based on a reference point within an image. Given a reference point which models 
the focus of visual attention, a surface is then constructed and flattened except along the existing 
real edges. The level sets of the surface function are able to connect the imaginary contours. Define 
the edge indicator function by [2*1] 

i 

g(x) = g(x 1 ,x 2 ) 



l + (|VG ff (x)*n(x)|//3) 2 



exp(-(£/q) 



Then the model is to minimize the g- weighted area of the surface S : x = (x%, X2) — * (x\, X2, 



mm / g[x J 
* la 



1 + ^ + <S>l 2 dx. 



Our model shares a similar computational formulation, but is more self-contained in terms of 
modeling and analysis since it is directly built on curves rather than on reference points and 
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surfaces. (We must point out that, in visual cognition, attention focuses do play important roles 
in a number of cognitive tasks 

Later in [SU], Zhu and Chan proposed a more complex level-set based variational model. Let 
u : Q — ► R be an image defined on the domain Q, with u = 1 on the objects and u = — 1 otherwise. 
The key component of Zhu and Chan's model is the signed distance function from solving the 
eikonal equation : 

^ = sign(u)(l- |W|), 

with the initial data d(x,t = 0) = u(x) = u(xi,X2)- Their variational model is then to minimize 
the following energy for the level-set function <j) which codes the desired illusory contours: 

E(<j)) = [ ((1 + nC{d)K + {d))\d\5{cp)\Vcp\ + \H{d)H {</>)) dx 

2\ (1) 

\Vcj>\8{4>)dx, 




V • 



W\ 



where 5 denotes Dirac's delta, H the Heaviside function, and CaC 1 differentiable cut-off function. 
The |(f|5(</>)|V(/>| term measures the distance from the zero level-set to the existent boundaries 
and C(d)K + (d)\d\5((/))\ V</>| measures the distance to concave corners. The second term of the first 
integral measures the overlap between the existent objects and the interior of eft. The last integral 
represents Euler's elastica energy which was first proposed by Mumford ^7], and later applied 
to geometric image inpainting by Chan, Kang, and Shen |2j, and Esedoglu and Shen {HJ. Zhu 
and Chan jSHj showed outstanding performance of this model for several general examples. Our 
proposed model is much less complex than this high-order geometric model, and is also independent 
of the signed distance function. The essence of our proposed model is to be able to capture the most 
salient features of illusory perception in a simple and analyzable manner, at the unsurprising cost 
of losing certain high-order details. (Zhu and Chan |31j later also employed prior shape information 
to capture illusory contours.) 

Mainly inspired by these pioneering works on quantitative modeling, we propose a new energy- 
based model for studying illusory contours. The model is directly formulated on admissible contours 
and does not depend upon reference points nor their associated surfaces. The model has a much 
lower order of complexity which however yields good leading-order approximation to most generic 
illusory contours (with exact capturing in the canonical cases of Kanizsa's triangle or square) . The 
relationship between this new model and the more complex models of Zhu and Chan is formally 
analogous to that between the first-order and second-order Taylor expansions in Calculus. Most 
remarkably, the lower complexity of the new model allows to almost completely characterize the 
structures of illusory contours, which provides a solid stepstone towards more general quantitative 
analysis in the future. To our best knowledge, such rigorous characterization has been unprece- 
dented in the literature. 

The second major feature of the new model is that in terms of computation, it can be efficiently 
implemented by classical active contours and level-set based algorithms of Osher and Sethian [22, 
121( 126 1 125], In particular, we introduce a key but natural step of supervision for capturing robustly 
illusory contours, which can also be considered as an innate component of the new model itself. 

The paper has been organized as follows. In Section [3 we develop and analyze the new model 
for illusory contours that emerge from any generic configuration of real objects. We are able to 
rigorously characterize the geometric structures of illusory contours when identified as the local 
minima of the proposed model. In Section^ the model is further extended to process configurations 
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with real contours. In Section |IJ an approximate or relaxed model is then developed based on the 
classical active-contour model of Kass, Witkin and Terzopoulos 12 , which allows the proposed 
model to be actually computed via asymptotic approximation. In Section the level-set method 
of Osher and Sethian |21| 1221 126j is employed to compute the stable local minima of the model, 
under the help of a key step of supervision. Finally, the details of numerical implementation as 
well as the computational performance of the proposed model are provided in Sectional 

2 Illusory contours as stable local minima and characterization 

In this section, we propose a contour-based energy model, and model illusory contours as the local 
minima of such an energy. The particular form of the energy allows detailed and rigorous analysis of 
the geometric and topological structures of admissible local minima. Such in-depth mathematical 
characterization is, to our best knowledge, unique in the literature. 

Note that a local minimum corresponds to an energy valley and is stable only to local pertur- 
bations. This differs our model from many other efforts in modeling illusory contours as global 
minima. Aiming at local instead of global minima has also been motivated by cognitive sensitiv- 
ities of illusory perception (of light, geometry, or objects, etc) to various factors including visual 
attention, focus points, orientations, and scaling, etc. Such visual factors may disrupt or erase an 
observer's ongoing illusory experience, while global minima often remain stable to these external 
perturbations. 

In later sections we then develop an efficient level-set-based computational algorithm that sup- 
ports stable stagnation at such local minima. 

2.1 Kinks, outer and inner spans 9, and a generic configuration Q 

Let Q denote the entire open image domain, which could be the complete plane M 2 or a finite 
rectangle area. Let Q = {Qi \ i = 1 : k} be a configuration of disjoint compact domains whose 
boundaries are piecewise-C 2 simple closed curves. Denote the boundaries by 



Figure 3: Left panel: a generic kink y with outer span 6 y > and inner span 6* = 2tt — 6 y > 0. 
Right panel: two examples of singular kinks or cusps; an inward cusp at z with 9* = and an 
outward cusp x with 9 X = 0. The current work discusses generic configurations containing only 
generic kinks. Perceptual effects of cusps have scarcely been experimented in visual cognition. 



dQ = dQ 1 udQ 2 u---udQ k . 




a generic kink y 



two singular kinks, or cusps, x and z 
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Definition 2.1 (A kink and its outer span 9 and inner span 9*). A point y G <9(J is said to 
be a kink if the two tangent lines from the two sides are not identical. If y is a kink, its outer span 
9 y is the angle between the two tangent lines when moving continuously from one arm to the other 
outside Q (see Fig. OJ). We further define 9* = 2ir — 9 y to be the inner span. For a non-degenerate 
kink y, both 9 y and 9* must be positive. 

A cusp is a singular kink point y for which either the outer span 9 y = or the inner span 9* = 
(see Fig. El . It will be interesting in vision psychology to understand the cognitive effects of cusps 
in various scales. In the current work we shall assume that the given configuration Q only contains 
generic kinks so that 9 y , 9* > 0. Such a configuration Q is said to be generic consequently. Almost 
all classical examples correspond to generic configurations. 

Definition 2.2 (Kink set K and minimum spans 9 m { n and ^™j n )- Let Q C Q be a generic 
configuration. Define its kink set K[Q] to be the collection of all kinks of Q, and its minimum outer 
span by 

#min = #min [Q] = mm{9 y \ y € K[Q]}, 

and minimum inner span by 

Cn = #min [Q] = ^{^ I V G # [Q]} = max { 2 ^ ~ *y I If G tf[Q]}. 

Since is compact and piecewise C 2 , K[Q] must be finite and both 9 mm and 0^ in are positive. 



2.2 Real and imaginary decomposition of a contour 

Let T be any piecewise C 2 simple closed curve in Q that does not intersect the interior of Q. Define 
its real and imaginary parts by 

r re = rndQ, r im = r\«9Q. (2) 

For an illusory contour T, the imaginary part corresponds precisely to the illusory interpolation 
(Fig. 0J). (The names "real" and "imaginary" have been formally motivated by complex analysis in 
which a complex number z can be decomposed into real and imaginary parts: z = z re + y/—l z- im .) 




Figure 4: Decomposition of V into real and imaginary parts (partial view of Kanizsa's triangle). 

Notice that since both V and dQ are compact and thus closed sets, r re must be closed in Q and 
T[ m relatively open in T. We shall only consider a curve T for which r; m contains finitely many 
connected components to avoid unnecessary technicalities: 

r im = 7 i 1 m u 7 2 m u---u 7 ™. 
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Since Ti m = T n Q c , the intersection of a simple closed (and thus locally connected) curve and an 
open domain, Ti m must be locally connected jTH]. Consequently, each connected component 7^ 
must be a connected curve segment with at most two endpoints. In the current work, we assume 
that the two endpoints are distinct. (Otherwise, the entire curve T becomes a loop which touches 
dQ only at a single point. Such a loop can easily shrink to decrease the energy to be proposed 
immediately next, and is hence irrelevant to the local minima.) Let drf denote the two boundary 
endpoints, and dT im = d^/l m U • • • U cfy-JJ,. 

2.3 Energy model and the structural theorem for local minima 

For a given generic configuration Q in a 2-D open domain f2, define the class of admissible curves 

C = {r I r is simple closed, piecewise C 2 with finitely many pieces, and T C\Q° = empty}. (3) 
For any admissible curve r G C, we define the energy 

E [T] = E D [T \a,0\ = a f ds + (3 ! ds, (4) 

where s denotes the arc-length parameter and < a <C (3. The subscript "o" in E Q signifies that 
this energy is for object-based configuration Q. In the next section, we will introduce an energy E c 
for contour-based configurations. Notice that E Q is the weighted sum of the lengths of r re and T im . 
We are interested in the behavior and characteristics of the local minimizers to E D , in particular, 
in how well they can quantify more familiar features of illusory contours in visual cognition, which 
have often been subconsciously processed by human observers. 

Lemma 2.3. Let T £ C be a local minimizer of E a . Then each connected component ofT im must 
be a straight line segment. 

Proof. Suppose otherwise that one connected component 7 of F; m is not a straight line segment. 
Then there must exist at least one point 6 6 7, within any of whose small neighborhoods, there 
can be found two points a, c 6 7 on the two sides of b such that {a, b, c} are not collinear. We then 
replace 7|r 0)C i by the straight line segment joining a and c. Since T and hence 7 are piecewise C 2 
and 7 C Q c (an open set), as long as the neighborhood is small enough, the straight line substitute 
does not intersect the rest of T nor Q. Denote the new curve after such a micro-surgery by T'. 
Then V £ C and ^[r'] < E \T] since the two only differ in between a and c. This contradicts 
to the assumption that T is a local minimizer since such micro-surgeries can occur in any local 
neighborhood of b. □ 

In the following analysis, the local perturbation of a junction point x 6 dT{ m along dQ (see 
x — > x e in Fig. EJ) has always been accomplished by a simple parallel translation of the attached 
imaginary ray 7i m . This is justified by the preceding lemma, as well as the fact that when the other 
endpoint y is relatively far away, the perturbed imaginary ray yx £ due to x — > x £ is parallel to yx 
in the first order (of e) within small neighborhoods of x (see Fig. Inland its caption). 

Lemma 2.4 (Two distinct imaginary rays cannot share a hinge). Suppose T £ C is a local 
minimizer to E a = E [-\a,(3], and 7 1 and "y 2 are two distinct connected components (which are 
straight line segments by the preceding lemma). Then there exists some critical ratio r c , such that 
as long as r = a/ (3 < r c , d^/ 1 n d*y 2 must be empty. 
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Figure 5: Local perturbation at a junction point x E dTi m C dQ can be approximated (in the first 
order of the displacement distance e) by a simple parallel translation of the attached imaginary ray 
7i m , due to Lemma l2.3l and the same mechanism that makes direct solar rays appear approximately 
parallel when reaching onto the Earth. Parallel shifting simplifies most computation. 

Proof. (See the companion illustration in Fig. EJ) Otherwise, assume that 7 1 and 7 2 share a hinge 
at a point z (i.e., the point O in the figure). Then z must be a real point, and one can assume that 
z E dQj for some unique j. 

Locally in a small diskette neighborhood of z, the two straight rays 7 1 and 7 2 segment the disk 
into two parts: an "inner" part that intersects with Qj and an "outer" part. Let 9 denote the outer 
span angle. 

First suppose 9 E (0,7r). Then any corner-cutting local deformation near z (as shown in the 
left panel of Figure EJ) can decrease energy E since the the real part of the energy is unchanged 
but the imaginary one shrinks. Thus this scenario cannot occur. 

Then one must have 9 > it. For dQj, similarly there are two real tangent rays at z, denoted by 
/j, 1 and /i 2 , so that the acute cone of /j, 1 and 7 1 contains neither of the other two rays. Let fa denote 
the acute angle between ji l and 7*, i = 1,2. Without loss of generality, assume that fa < fa. Since 

<Pl + 4>2<(27T-9)-9* min <7T-9; nin , 

one has 

fa = min(fa,fa)<^- e -^. 

Then as shown in the right panel of Fig. a small parallel sliding (with perpendicular distance e) 
of the root of the imaginary ray 7 1 along the real edge of dQj in the direction of /i 1 will introduce 
an energy increment of: 

e e e(3 (a \ 

sm (p\ tan fa sin fa \ p J 

Define r c = cos(7r/2 — ^ min /2) = sin(^ in /2) > 0. Then as long as r = a/ [3 < r c , the leading 
order of energy increment is in fact negative, which again contradicts to the given assumption. To 
conclude, as long as r < r c , two distinct imaginary line segments cannot share a hinge. □ 

Definition 2.5 (Junction set J[F] and a turn (p z at a junction point z). For a minimizer 
r E C, we define its junction set to be 

J[r] = dT{ m , i.e., all endpoints of imaginary segments. 
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Figure 6: Companion illustration for the proof of Lemma [2.41 Two distinct imaginary rays 7^ and 
Tim (°f a l° ca l minimum contour T) cannot share a common hinge O ( Lemma 12. 4|) . Left panel: 
if 6 < it, a local perturbation of AOB to AA'B'B decreases the total energy E a . Right panel: if 
9 > ir, a local perturbation of AOB to A'O'OB still decreases the total energy (see the computation 
in the proof of Lemma l2.4|) . 



By definition, one must have J\T] C dQ. For any junction point z S J\P], in one direction T goes 
to a uniquely associated imaginary ray, and let t; m = U m (z) denote this outgoing unit vector of 
orientation, and in the other direction comes a curve segment of r re (according to the preceding 
lemma) and let t re = t re (z) denote this incoming unit vector of orientation (see Fig. [7J). The turn 
at z is then defined to be the angle: 

4> z = arccos(t im • t re ) £ [0,7r]. 




Figure 7: A permissible geometric layout at a junction point z G J[T] with the turn cp z < tt/2 and 
idle turn <p z > according to the results established in Lemmas 12.61 and 12.91 

Lemma 2.6 (Acute turns at junction points). Suppose V £ C is a local minimum of E Q and 
z £ J[r] is a junction point. Then the turn 4> z < tt/2. 

Proof. Suppose otherwise that (f> z > tt/2. Then if one slides the root z of the imaginary ray 7; m 
along the incoming part of r re while keeping the imaginary rays parallel, as well as other parts of 
r intact, as in the left panel of Fig. |HJ the leading order energy increment becomes: 



tan(7r — (j) z ) sin(-7r — <j) z ) 
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which contradicts to the assumption of T being a local minimum. (Notice that in the critical case 
when <fi z = it/ 2, the first term degenerates into a second-order change 0(e 2 ). Thus the result still 
holds due to the second term.) □ 



This lemma has an intuitive interpretation in terms of visual cognition. Illusory contours more 
or less are smooth and straight extrapolations of existing real boundaries jUIE], an d consequently 
the turns at the junction points should generally be as small as possible. 

Definition 2.7 (Maximum turn of a local-minimum contour T). Suppose r £ C is a local 
minimum of E Q . Define the maximum turn (/> max of V by 

</>max = max{0 2 I z £ J[r]}. 

Then the preceding lemma implies that </> m ax < it/2. 

Definition 2.8 (Idle angle ldlc at a junction point). Let r £ C be a local minimum to E a , and 
z £ J[r] a junction point. Define the imaginary tangent £i m = U m (z) as in Definition 12.51 Further 
define the idle tangent t;di c = iidic^) to be the unit outgoing tangent vector of dQ \ T TC (i.e., the 
idle real boundaries that do not participate in the formation of T) at z (see Fig. |HJ. Define the idle 
angle at z by 

(pf le = arccos(t id ie • *im)- 




Figure 8: Companion illustration for the proofs of Lemmas 12.61 and 12.91 Left panel: the turn <j) z 
at a junction point z £ J\T] cannot be blunt, otherwise the local perturbation of z to z' and 7i m to 
decreases both the real and imaginary energies in E a . Right panel: the idle turn c/>l, dle cannot 
be acute for sufficiently small a/f3, otherwise the perturbation shown will cause a net drop in the 
total energy E Q (see the computation in the proof of Lemma l2.9|) . 



Lemma 2.9 (Behavior of idle angles at junctions). Suppose V £ C is a local minimum to 
E {- | a,/3] for any a,/3 > ; and z £ J[F] is one of its junction point. Then </>^ dle > tt/2. 

Proof. Suppose otherwise that ^ dle < tt/2. Then if one slides the root z of the imaginary ray 7; m 
along the outgoing idle part of dQ \ T te while keeping the imaginary rays parallel, as well as other 
parts of r intact, as in the right panel of Fig. |S1 the leading order energy increment becomes: 

a . e rM - (3 = . £(3 r „ (% - cos(4 dle )^) . (5) 

sin^ dle M tan^ dle sin0 ldle \(3 ) 
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Since T is a local minimum for any a, (3 > 0, one must have 

-cos(^ dl °) > 0, or ^ dle >vr/2. 

□ 

Combining all the characterizations established in the lemmas of this section, we are ready to 
state the main theorem of this paper. 

Theorem 2.10 (Characterization of a local minimum). Let Q be a generic configuration on 
an open domain and $min ? 

^min > minimum outer and inner spans. Suppose a given simple 
closed curve T € C satisfies the following structural conditions: 

(i) (Imaginary behavior) each connected component 7i m ofT{ m is a straight line segment, and no 
two distinct components share a common hinge; 

(ii) (Junction behavior) at any junction point z £ J\P], the turn (f> z < ir/2, and the idle angle 
(j)f lc > vr/2. 

Let 6 

max denote the maximum turn on J[T]. Then there exists a critical ratio r c — r c (9 m \ n , (/> max ) < 
1, such that for any a and (3 with r = a/f3 < r c , T is a local minimum to the energy E Q [- \ a,0\. 

Proof. We shall only outline the main steps of the proof and leave the details to the reader, which 
are very similar to the proofs of the above ensemble of lemmas. Let J\T] denote the junction set 
of such a given T, and define 

r° e = r re \ j[T}. 

Then T is segmented into three distinct parts: 

r = r im ur r ° e u j[r], 

each of which leads to a distinct energy variation pattern due to local admissible perturbations. 

First, it is easy to see that under Condition (i) any local perturbations at a point of r; m will 
increase the energy. 

Next, r° e can be further partitioned into kink points r° o ni^[(5] and the majority rest of smooth 
points. At a smooth point y, any generic local variation drifts a real local segment 7 into Q c and 
converts it into an imaginary one, and hence introduces a net leading-order energy increment 
of (P — a) J ds, which is always positive for r = a/ [3 < 1. At a kink point y on T° e , one defines 
r\ = sin# max /2 > 0. Then simple trigonometric calculus shows that for any a and [3 with a/ (3 < n, 
any local admissible displacement of the kink point always increases the energy. 

Finally, one turns to the local perturbations at any junction point z S J\P]- By Condition (ii) 
on the acute turns, one must have that the maximum turn max < tt/2. Define r^ = cos</> max > 0. 
Then trigonometric calculus shows that any local drifting of z into the incoming real contour T re 
will increase the energy as long as a/ (3 < r2- On the other hand, any local drifting of z into the idle 
real edge dQ \ T ie always increases energy due to Condition (ii) on ^^ dlc > ir/2, for any a, [3 > 0. 
The details are very similar to the proofs of Lemmas 12.61 and 12.91 

In combination, one defines the critical ratio r c = min(ri, r2, 1). Then for any a and (3 with 
a/ (3 < r c , the above analysis in all the three scenarios shows that the energy always increases under 
any local perturbations of T. Thus T must be a local minimum to E a [- \ a, 0\. □ 
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By this theorem, it is straightforward to establish the following corollary, which demonstrates 
that local minima of E are indeed reasonable models for illusory contours, at least in terms of the 
leading-order approximation. 

Corollary 2.11. Both Kanizsa's triangle and square are local minimizers to E a [T \ a, f3] for suffi- 
ciently small a/ (5. 

Remark 2.12. In the preceding main theorem, only the condition <^ dl ° > tt/2 could be relaxed to 
c^ dle > tt/2 — 5 for some constant 5 = 6(a, (3) > with 5 — > as r = q//3 — > (due to Eqn. ©). The 
preceding lemmas show that all the other conditions are necessary and cannot be further relaxed. 

3 Real contour bundles and object-contour mixtures 

In this section, we briefly discuss contour-based illusory contours, as well as those arising from a 
mixture configuration of both objects and contours. Computationally, however, it can be handled 
by exactly the same scheme for the object-based model, as to be numerically demonstrated later 
on. 




Figure 9: Decomposing a candidate contour T into its real part r re and imaginary part Ti m . 

Let L = {Zj}™ =1 be a bundle of disjoint compact contour segments on a given open image domain 
Q. To be distinguished from illusory contours, each ij shall be called a real contour (see Fig. |5J). 
Let dli = {ctj, bi} denote the two endpoints of Zj. Define 

dL = U? =1 dk = UjL^Oi, h}, L° = L\ 8L. 

As for object-based illusory contours, we define the class of contour-based admissible contours by 

C c = {T | T is simple closed, piecewise C 2 , and m L° =empty}. 

For each admissible contour T £ C c , we define, in the same spirit of the previous object-based case, 
the real- imaginary decomposition (Fig. |SJ): 

r re = rndL, r im = r\dL = r\r re . 

As a result, the previous object-based energy E Q is modified to the following contour-based: 

E C [T] = -aH°(T re ) +f3 [ ds = -aH°(T Te ) + (3 [ ds, (6) 

with a, (3 > 0. Here 7i° denotes the 0-dimensional Hausdorff measure, or equivalently, the atomic 
counting measure. 
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The major difference of E c from the previous object-based energy E Q resides in the first negative 
term. Under this new formulation, any real end point from dL acts like an energy basin, in order 
to be able to anchor prospective illusory contours. 

The local minimizers of E c are much easier to characterize. 

Theorem 3.1. r E C c is a local minimizer with a non-empty imaginary part T im if and only if it 
is a polygon whose vertices are all anchored within dL. 

Proof. First suppose T is a local minimizer with a non-empty imaginary part Ti m . For any connected 
component 7 of Ti m , the same argument as in Lemma l2.HI applies since a local perturbation of 7 
only modifies the length integral in the /3-term. Thus 7 must be a straight line segment. Since 7 
is arbitrary, T € C c has to be a polygon whose vertices are anchored within dL. 

Conversely, suppose r £ C c is a polygon whose vertices belong to dL. Any local variation within 
a side 7 of T increases the /3-term line integral while leaving the atomic a-term unchanged, since 
7 is straight. On the other hand, any nonzero local perturbation of a vertex of T will drift it into 
Q\L (since the perturbation must be kept within the admissible class C c ), and consequently cause 
a leading-order energy increment of a. Hence T is a local minimizer, and the proof is complete. □ 




Figure 10: Decomposition of a candidate curve T in an object-contour mixture configuration. 

Finally, some examples in the literature also involve illusory contours from a mixture configura- 
tion of both objects and contours. For this reason, we consider a mixture model briefly. Consider 
an image which is composed of compact and mutually disjoint real objects Q and real contours 
L. Let r be an admissible piecewise C 2 simple closed curve in Q that does not intersect Q° U L° 
(defined previously). We define the real and imaginary part of V as follows: 

r re = r re . ur re . c = (rndQ)u (rndL), 
r im = r\r rc = r\(dQudL). 

Notice that here the real part has been further categorized into object-based real part r re o and 
contour-based real part r re _ c . We then define a mixture energy functional E m in the form of: 

E m [T] =a I ds- a c H°(T re _ c ) + (5 j ds. 

where < a <C (3 and a c > 0. 

The characterization of a local minimizer to this mixture energy can be readily obtained from 
a straightforward combination of the previous two major theorems. In particular, for example, if T 
is a local minimizer, then two distinct connected components 7 1 and 7 2 of the imaginary part can 
share a common hinge, but the hinge point must belong to dL and not to dQ. 
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Next we discuss how to compute these models in real applications. In particular, we will 
demonstrate numerically that for practical applications, a single relaxed model is sufficient to 
handle all the previous three theoretical models of E ,E C , and E m . 



4 Relaxation of the model: approximation by active contours 

The main models have been built upon the real-imaginary decomposition of any candidate contour 
r, which in turn depends upon the identifiability of the real edge set dQ. Hence arise two major 
practical issues for implementing these models realistically. First, for general images, one has to 
perform a pre-processing of edge detection in order to obtain dQ, which is easier for binary images 
but highly nontrivial for general continuous-tone images. Second, even provided with dQ, it is well 
known that direct computation of curves is highly challenging due to the densening or coarsening 
effects of discrete points as well as potential topological changes like splitting and merging. 

The current section resolves the first issue, while the next section will address the second. 

The main result of this section is to show that the edge-detection process can be implicitly 
integrated into an approximate model, which surprisingly turns out to be the more familiar ac- 
tive contour model El 021 El- That is, the proposed new model for illusory contours can be 
asymptotically approximated by active contours when restricted in the class of admissible contours 
C. 

First consider an ideal binary image u(x) = Xq{ x )i i- e -> the characteristic function of a generic 
configuration Q. Let r](x) be any C°° positive function which is compactly supported on the unit 
disk in the plane, and for any scale a > 0, define the mollifier r q a {x) = -^T](— ), and the mollification 
u a = u * r} a . Since Q is assumed to be compact in the open image domain f2, u a is well defined as 
long as a is smaller than the distance dist(Q, d£l). 

Then for any admissible curve T £ C, we introduce the following approximate energy model 

E a [T\u)=a f^ds + (3 J g(\V UtT (s)\)ds, (7) 

where g(t) = pAp for some fixed parameter A > 0. This is in fact precisely the classical active 
contour model, where g(\S/u a (s)\) is an edge signature distribution, or a soft edge indicator [T2] . 
The next theorem explains the relation between E Q and E a . 



Theorem 4.1. For any admissible curve V E C, one has / — — — r^ds — ► / ds as a — > 0. 

J r l + X\Vu a \ 2 7r im 

Proof. Denote B a = {x £ fi| dist(x, dQ) < a}. Then u a = 1 in Q \ B a and u a = in Q c \ B a . As 
a result, Vii CT = and 1+x ^ u p = 1 in O \ B a . Since as a — > 0, B a shrinks to dQ, one has 

i 

1 as a — ► pointwise in O \ dQ. (8) 



l + A|V?v| 2 

Since u is a characteristic function, one must have | Vu CT | — > oo as a — > along the piecewise smooth 
edge dQ, and 

In combination of (jHJ and Q, one has 

Sl + A|W| 2=XfA ° Q ' 
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pointwise on the entire image domain 0,. Since r £ C is piecewise smooth, simple closed, and Tf]Q° 
is empty, by applying Lebesgue 's dominated convergence theorem on V with respect to its arclength 
measure ds\r [2], one has, as a — ► 0, 

/ i+aiWi 2 * - L x ^ ds = L\ Q ds = L ds - 

This completes the proof. □ 
Consequently, as a — > 0, one has for any admissible contour T £ C, 

E a [T\u] — >a ds + (a + (3) ds. 

In particular, the current parameter pair (a, a+0) corresponds to the original parameter pair (a, f3) 
in E a . The approximate energy E a is, however, computationally much more tractable since the 
practically difficult decomposition V = T Te U Ti m is replaced by much simpler function evaluations 
of g and u a . This technique is well known in the literature of active contours jSJ . 

Furthermore, the active-contour approximation E a , rigorously established above for binary im- 
ages, is now readily applicable to general images which could have continuous tones or have been 
degraded by moderate noises. 

Remark 4.2. Due to mollification, 1-D real contours in a mixture configuration are mollified to 2-D 
stripes. As a result, our later computational results show that the relaxed active-contour model 
works equally well for contour configurations or object-contour mixtures. 

Remark 4.3. It is well known that classical (non-region-based) active contours are often easily 
trapped in local minima |BJ. This no longer becomes an issue herein since illusory contours have 
been actually modelled as local minima, as extensively developed in the preceding sections. That 
is, it would be bad news if the active contour model E a never stably stagnates at local minima. 

The key question becomes how to compute robustly those local minima that correspond to 
meaningful illusory contours in cognition. This issue is resolved by the supervised level-set compu- 
tation, which is the main theme of the next section. 

5 Supervised level-set-based robust computation 

In this section, we derive the level set formulation to the approximate model ((7J via classical active 
contours, and introduce a simple but key step of supervision to be able to extract illusory contours 
as stable local minima. 

5.1 Derivation of the Euler-Lagrange equation 

The following brief derivation of the Euler-Lagrange equation is canonical (see, e.g., [HIE]) and 
makes the current paper more self-contained. 

To derive the formal Euler-Lagrange equation, the candidate contour T is assumed to be at 
least C 2 so that second-order geometric features like curvature k are well defined. We denote by 
<&(x) = ^(x \ u,a) = g(\Vu a (x)\) for convenience. Then 

E[T\u]=a J ds + pj <5>(s)ds, 
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where ds denotes the arc-length element of T. Since T is assumed to be C 2 , any smooth perturbation 
of T r of r can be represented as a displacement along the normal n: 

c r (s) = c(s) + r5c(s) = c(s) + rh(s)n(s), r e (— e,e), h = 0(1). 

Then by Frenet's frame formula [S], n'(s) = —K.(s)t(s), and 

c' r (s) = (1 - rh(s)K(s))t(s) + rh'(s)n(s), 



\c' r (s)\ = yJ{l-rh{s)K{s)) 2 +r 2 h'{s) 2 . 

Suppose s E [a, b] is the domain of c and s denotes the arc-length parameter of c r so that ds 
\c' r (s)\ds. Then 



ds 



d f f b d 

31 / d7s = / 7nK( s )\ ds 



dr 



dr 

d_ 

dr 



ds 



=0 JT r 



c' r (s)\ds. 

J a 

b -(1 - rh(s)n(s))h(s)K(s) + rh'(s) 2 
a ^(1 - rh(s)K(s)) 2 + r 2 h'{sY 

b 

—h(s)K(s)ds 
—K(s)n(s) ■ h(s)n(s)ds. 



ds. 



(10) 



On the other hand, for the second term, by noticing that 

JU(c,.(s)) = V<D • ^c r (s) = V<& • h(s)n(s) = h(s)^(c(s)), 



one has 



± f <5> d ~s= f b |_$( Cr ( s ))| c ;( S )|d S + f\(c r (s))^-\c f r (s)\ds, 



t/r 



®ds 



r=0 JT r 

In combination of (|1U|) and (| 1 lj) . one has 



(9$ 

— (c(s))ds - / <f>(c(s))h(s)K(s)ds. 
on ' 



(11) 



d_ 

dr 



E a \T r \u] = — a K(s)n(s) ■ 5c(s)ds 
=o Jr 

-13 J ($(s)K(s)n(s) - ^(s)n(s)\ ■ 5c(s)ds. 

As a result, after introducing t as the artificial evolution time, we arrive at the gradient-descent 
equation for £? a [T|u]: 

3c 

= ( a + pq>) Kn -p^ n n. (12) 

at 

In particular, if one defines 

G(x) = G(x | a,P,a,u) = a + /?<3?(x) = a + 0g(\Vu a (x)\), 
then (j!2[) simplifies to 

dc 

— = Gku - G n n. (13) 
This is the canonical "snake" equation in its original form (see, e.g., pi I12|). 
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5.2 Supervised computation via the level-set method 



The level-set method of Osher and Sethian |221 1211 126j is a powerful computational tool to handle 
curve and interface motions, especially those that intrinsically involve topological changes. The 
above active contour equation has a simple formulation under the level-set representation. 

Suppose the evolving contour c(t, s) is always identified as the zero-level contour of an evolving 
level-set function <p(t,x), which is smooth or at least Lipschitz continuous: 



0(t,c(i,s)) = 0, Vt > 0. 



(14) 



Assume for convenience that <f> is positive in the interior of c(t,s). By differentiating (|14j) with 
respect to t, one has 

d<t> /„ , dc N 



eft 



at 



o. 



Recall that for an implicit contour, the curvature, and tangent and normal vectors can be repre- 
sented as follows: 



K 



-V- 



V0 



|V^| 



V0 X 



n 



(15) 



Thus the active-contour model (|13j) allows the following level-set representation: 



w + < v *--° v 



V0 

1W 



G r 



\ 

IWI / 



0. 



It can be further simplified to 



at 



|V0|GV 



+ |V0|G„. 



Since VG = Gti + G n n, one has 



VG- V0= |V0|(VG-n) = |V0|G n , 



and consequently, 



G 



(16) 



(17) 



(18) 



In the variational-PDE literature, this evolutionary partial differential equation is integrated via 
the Neumann boundary condition §^ = along 317. 

Since in our model, illusory contours have been modelled as local minima, the initial contour or 
the initial configuration of the level-set function becomes important. In the numerical computation, 
we have chosen the initial function 4>q such that 4>q = 1 in VL e and —1 otherwise. Here fi e denotes 
an open sub-domain of SI such that the Hausdorff distance between f2 and £l £ is less than e. That 
is, £l e is a slightly contracted version of £1. As long as the targeted illusory contour is a compact 
set on Q, £l e can capture it within for small enough e. 

Finally, unlike the classical active-contour computation whose goal is to capture real boundaries 
of existent objects, the active-contour model E a in the present context approximates our original 
contour model E a and hence is targeted at capturing illusory contours. There is a side condition 
which has been stated theoretically throughout the earlier discussion but not yet incorporated into 
the level-set formulation. In the definition of the admissible class C of candidate illusory contours, 
we have required that any admissible contour T must lie outside the object configuration Q, i.e., 
that r n Q° is empty. 
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Classical level-set functions for active contours ideally prefer <f> > on Q in order to capture real 
edges and real objects, but the above discussion shows that for the computing of illusory contours, 
ideally one prefers <j> < on the object configuration Q. Thus we impose the following supervision 
condition for the level-set computation: 



<j)(t,x) = -d Q (x) 



x G Q°, t>0, 



where cIq(x) = dist(x,<9Q) is the absolute distance function. In reality, for computation on a 
discrete image lattice, the supervision can be conveniently replaced by the less complex one: 



<f)(t,x) 



-1, 



x G Q°, t > 0. 



On one hand, it substantially lessens the computational burden without turning to the eikonal 
equation for cIq(x), and on the other, it provides a cheaper way to minimizing the second term in 
Zhu and Chan's model (0) without turning to either the variational formulation or its associated 
Euler-Lagrange equation. To conclude, the last supervision condition is an efficient tool to reinforce 
the contour admissibility condition T D Q° = empty" in the original exact model. 



6 Implementation details and computational examples 

In this last section, we explain the numerical implementation and discuss several generic numerical 
examples. The equation and conditions in the preceding section E2] can be reiterated by 



0(0, x) = (/) (x), 



V0 



for (t, x) G (0, oo) x O, 



(19) 



-- = for (t, x) G (0, oo) x dtt, 
av 



k <f>(t,x) 



-1 for (t,x) G (0,oo) x Q°, 



where G(x) = a + f3g(\Vu a (x)\). 
Let 

x = (xi,X2), and t m = mAt. For simplicity, we assume the spacial sizes coincide: Ax± = Ax2 
Then, 



%j be the approximation to the value (t>(t m , xj, where x\ = iAxi, Xj = jAx2 with 

1. 



t) m+1 



where | • | e is defined by 



{a + f3g(\(Vu a )^\)) 



Vs 2 + e 2 



V^ m 



1,3 



'■J 



(20) 



for some positive e <C 1. This simple regularization technique helps the computation of the de- 
nominator |V(/>| when it is close to be singular (see, e.g., (23 EHl EZj ) • We now focus on the spatial 
discretization of the right hand side at a fixed time t m , and shall leave out the superscript m 
for convenience. We have employed central differencing for differential discretization, and mean 
filtering for interpolation. Write 
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Based on central differencing at the half points of the Cartesian lattice, one has 

8 



dx M dy l '° 



1+2,3 1 -5>J/ V % >3+2 hJ—sJ 

Thus we need to specify the half-point values in l|21|). For example, at (i + 5, j), one has 

—6. 1. 1 
dx l+ 2'i 



(21) 



d_ 

dy 



+ 



~ 9i,j-l 



To satisfy the supervision condition 4>(t,x) = —1 for (t,x) 6 (0, 00) x Q°, we reset <p T [ l j' 1 = —1 on 
Q° after each discrete evolution (|2"U|) . For more details on numerical discretization, we also refer 
the reader to 0. 

Shown in Fig. ^2 through ^3 are computational outputs on some generic examples of illusory 
contours, including the well known classical examples of Kanizsa's triangle (Fig. 111(1 and square 

(Fig.ua). 

Fig. EH shows an example of illusory contours from a configuration of contour bundles, as 
theoretically modelled in Section |S1 Due to the mollification effect, the supervised level-set-based 
active contour model, which has been mainly developed for o&jerf-based illusory contours, works 
equally well for contour-based illusory contours in terms of computation. 

Fig. ^] shows a computed illusory contour emerging from a more complex configuration Q of 
objects. The small hump remaining in the final output is not due to numerical defection but to the 
competition between the two terms in the model E Q , or equivalently, between a and (3. 

Finally, Fig. El shows the first-order nature of our proposed model. In order to be able to 
output smoother illusory shapes, high order geometric features such as the curvature have to be 
incorporated, as in Zhu and Chan's earlier works |3Ul I31j . The tradeoff is that comprehensive 
analytical results as in Section El would be extremely difficult to develop, which is well known in 
the literature of variational modelling involving curvatures [21 117j. 

The captions of the figures also provide further on-site explanation and discussion. 




Figure 11: Kanizsa' illusory triangle as a stable local minimum to the proposed contour energy. 
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Figure 12: Kanizsa' illusory square as a stable local minimum and the stagnation process. 







Figure 13: An illusory square emerges from a configuration of real contour bundles. The object- 
based supervised level-set computation works equally well for contour-based illusory contours due 
to the mollification operator applied to a given image u. 
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